1. Field of the Invention
This invention relates to image processing, particularly to an image processing apparatus which can be used as an “electronic makeup” apparatus providing high quality and natural looking images in a field where sophisticated works are required in both the contents and representation, such as television dramas and movies.
2. Description of the Related Art
It is known that by applying a two-dimensional ε-filter (for example, refer to Harajima and others “ε-Separating Nonlinear Digital Filter and Its Applications” IEICE (The Institute of Electronics, Information and Communication Engineers) 1982, 4,J65-A, No.4, pp297-304) to an image of a human face, neck, hand, etc., a “smooth skin” image with wrinkles and spots removed or decreased can be obtained (refer to Arakawa and others “Color Face Image Processing by Vector ε-Filter˜Removal of Wrinkles˜” March 1998, 1998 General Conference of IEICE D-11-143, PP143-). This will be hereinafter referred to as conversion to “smooth skin.” This is based on the fact that small-amplitude light and dark change is smoothed by “separating and suppressing small-amplitude high-frequency noise components in an image”, which is a function of the two-dimensional ε-filter.
The ε-filter was originally designed for the purpose of separating and removing the small-amplitude high-frequency noise components superimposed on a signal waveform.
A low-pass filter (LPF) that is usually used for noise removal not only suppresses noise components, but also degrades edges of signals. Thus if it is applied to an image, the low-pass filter has the disadvantage of blurring the whole image. However, as shown in the relationship between the input and output signals of the ε-filter in FIG. 1, the ε-filter has a characteristic of flattening only small-amplitude level changes in a signal waveform, and has a feature of scarcely impairing sharpness of the whole image because the edges which have steep level changes are preserved.
Letting an input signal sequence be x (m, n), the output signal y (m, n) of the two-dimensional ε-filter is represented by the following expression (1):                               y          ⁡                      (                          m              ,              n                        )                          =                              x            ⁡                          (                              m                ,                n                            )                                -                                    ∑              i                        ⁢                                                   ⁢                                          ∑                j                            ⁢                                                a                                      i                    ,                    j                                                  ·                                  F                  ⁡                                      (                                                                  x                        ⁡                                                  (                                                      m                            ,                            n                                                    )                                                                    -                                              x                        ⁡                                                  (                                                                                    m                              +                              i                                                        ,                                                          n                              +                              j                                                                                )                                                                                      )                                                                                                          (        1        )            where ai,j is a weight coefficient. If the filter size is (2M+1) X (2N+1), it will satisfy the following expression:                                           ∑                          i              =                              -                M                                      M                    ⁢                                           ⁢                                    ∑                              j                =                                  -                  N                                            N                        ⁢                                                   ⁢                          a                              i                ,                j                                                    =        1                            (        2        )            
The function F(x) represented in expression (1) in a nonlinear function shown in a graph of FIG. 2 and F(x)=0 when |x|>εo.
Throughout the specification, the value of εo will be referred to as ε value.
FIG. 3 shows a basic configuration of the two-dimensional ε-filter.
In FIG. 3, the solid line frame denoted by numeral 1 indicates a calculation section of a small-amplitude high-frequency noise component u (m, n) (the second term on the right side in expression (1)). Output from the calculation section is subtracted from the input signal sequence x (m, n), whereby the output signal sequence y (m, n) with the small-amplitude noise component suppressed is provided.
By applying ε-filter to an image of a human face, the face image can be converted into a beautiful face having “smooth skin” with wrinkles and spots removed or decreased. The wrinkles and spots are not so-called noise, but involve comparatively small-amplitude light and dark changes in an image. The wrinkles and spots can be made inconspicuous, by smoothing these small-amplitude light and dark changes with the small-amplitude level change suppressing function of the ε-filter.
Here, only the small-amplitude level changes such as the wrinkles and spots are flattened and steep level changes such as boundaries of pupils, eyelids, eyebrows, etc., are kept, so that sharpness of the whole image is scarcely impaired and conversion to “smooth skin” can be accomplished.
Conversion to “smooth skin” is intended for skin areas strictly. However, if the whole image is processed uniformly through the ε filter, small-amplitude level changes of peripheral images are also suppressed and as a result, delicate light and dark patterns of hairs, clothes, backgrounds, etc., are also flattened, resulting in an image where original details and texture are impaired.
This is fatal in applications wherein high quality is demanded for the whole image such as television, movies, etc. However, the ε filter is made to selectively act only on the skin color area of an image, whereby conversion to “smooth skin” can be accomplished without impairing any details or texture of peripheral images. To do this, a technique called “chromakey” (technique of electronically discriminating a specific color area in an image from others and applying image processing such as filtering on y to the area restrictedly), a well-used device in the television technical field, may be adopted.
As described above, conversion to “smooth skin” through the ε filter has the potential for use as “electronic makeup” in television drama and movie production, etc. However, to make the most of the ε filter in the field where there are various demands for image processing and requirements for high quality of the whole image, the following problems to be resolved exist:
Conversion to “smooth skin” makes a skin image seem to be smooth, and thus also produces the “rejuvenation” effect as video representation. For an elderly actress to act as a girl such as in a drama of a biography, etc., although there are existing points to be improved (which will be pointed out later), conversion to “smooth skin” is a promising technique to be used as an electronic makeup technique where “rejuvenating skin” is obtained by performing image processing.
As described above, conversion to “smooth skin” can be realized using the ε filter. On the other hand, for playing an aged person drastically exceeding the actual age such as in a television drama or a movie, an electronic makeup technique for producing an “old” effect such as “wrinkle enhancement” is also demanded. However, hitherto, it has been impossible to realize “wrinkle enhancement” which can also be said to be the opposite effect to conversion to “smooth skin.”
In conversion to “smooth skin” using the ε filter, the degree (strength) of conversion to “smooth skin” can be easily adjusted simply by changing one parameter (εo in FIG. 2) in response to the strength of the wrinkles and spots to be hidden. Although the whole skin becomes smooth as the degree of conversion to “smooth skin” is increased, the grain and texture of the skin are lost and “plastic” texture rather than “human skin” results, leading to an image poor in reality. To avoid this, if conversion to “smooth skin” is weakened, the natural look is recovered, but the wrinkles and spots to be hidden appear; this is a dilemma.
In other words, a solution to “provide an image where the grain and texture of the skin are preserved while hiding annoying wrinkles and spots” becomes necessary.
Further, assuming that the above-described “wrinkle enhancement” can be realized, it is desired to provide a “wrinkles enhanced” image with the grain of the skin kept in the original state rather than simple “wrinkle enhancement,” needless to say.